Using a generic classifier to train a personalized classifier for wearable devices

ABSTRACT

Systems and methods may provide for using one or more generic classifiers to generate self-training data based on a first plurality of events associated with a device, and training a personal classifier based on the self-training data. Additionally, the one or more generic classifiers and the personal classifier to may be used to generate validation data based on a second plurality of events associated with the device. In one example, the personal classifier is substituted for the one or more generic classifiers if the validation data indicates that the personal classifier satisfies a confidence condition relative to the one or more generic classifiers.

CROSS-REFERENCE WITH RELATED APPLICATIONS

The present application claims the benefit of priority to U.S.Non-Provisional patent application Ser. No. 14/318,555 filed on Jun. 27,2014.

TECHNICAL FIELD

Embodiments generally relate to classifiers. More particularly,embodiments relate to using a generic classifier to train a personalizedclassifier for devices such as wearable devices.

BACKGROUND

A main source of complexity and/or difficulty in performing eventcategorization tasks (e.g., physical activity recognition, gesturerecognition, etc.) may result from large variations among sample databelonging to the same category. Such data variations may be dueindividual differences (e.g., different people walk differently) and/orsensing variations (e.g., phone in the hand versus phone in the pocketof the same person detecting different signal patterns for the samewalking activity).

BRIEF DESCRIPTION OF THE DRAWINGS

The various advantages of the embodiments will become apparent to oneskilled in the art by reading the following specification and appendedclaims, and by referencing the following drawings, in which:

FIG. 1 is a block diagram of an example of a classification approachaccording to an embodiment;

FIG. 2 is a flowchart of an example of a method of operating a deviceaccording to an embodiment;

FIG. 3 is a flowchart of a more detailed example of a method ofoperating a device according to an embodiment;

FIG. 4 is a block diagram of an example of a logic architectureaccording to an embodiment; and

FIG. 5 is a block diagram of an example of a device according to anembodiment.

DESCRIPTION OF EMBODIMENTS

Turning now to FIG. 1, a classification approach is shown in which auser 10 interacts with a device 12 such as, for example, a wearabledevice (e.g., electronic jewelry, clothing, footwear, eyewear, headwear,etc.), notebook computer, tablet computer, convertible tablet, personaldigital assistant (PDA), mobile Internet device (MID), and so forth. Theinteraction between the user 10 and the device 12 may take the form of,for example, gestures (e.g., hand, head and/or body gestures) made bythe user 10 and detected by a camera, touch screen (e.g., SWYPE dataentry), accelerometer, gyroscope, etc., of the device 12, or otherphysical activity (e.g., walking) conducted by the user 10 while incontact with the device 12. Those interactions may be detected as events18 (18 a-18 c) that may be automatically categorized/classified by thedevice 12 and used to perform other operations such as, for example,cursor movements, application launches, networking communications, andso forth, in real-time.

The illustrated device 12 includes one or more generic classifiers 14and a personal classifier 16. In general, the generic classifiers 14 maybe software solutions that are trained offline for a plurality ofindividuals (e.g., a large population containing different demographicsand/or types of individuals) and may therefore not be tailored to thebehavior, patterns and/or movement of the particular user 10. In theillustrated example, the generic classifiers 14 are used online (e.g.,“on the fly”), via arrows 22, to generate self-training data 20 based ona first plurality of events 18 a associated with the device 12.Additionally, the personal classifier 16 may be trained online, viaarrow 24 based on the self-training data generated by the genericclassifiers 14.

Because the illustrated self-training data 20 results from interactionsconducted only by the user 10, once trained the personal classifier 16may be tailored to the behavior, patterns, and/or movement of the user10 in particular (e.g., and not the general population at large).Accordingly, the personal classifier 16 may be highly accurate inclassifying events associated with the interactions conducted by theuser 10. Indeed, if the device 12 is a wearable device such as, forexample, a watch or eyewear, the accuracy of the personal classifier 16may be even greater because wearable devices may typically be attachedto the body in predictable ways. Therefore, given the intended usage ofthe device 12, the placement of the device 12 may have negligiblevariability with respect to the particular user 10.

Moreover, the personal classifier 16 may be implemented as a hardwaresolution (e.g., programmable pattern recognition neural network) havingan accelerated responsiveness that is suitable for real-timeapplications. For example, one such hardware solution might include asilicon implementation of an associative memory having an array ofneurons, wherein each neuron stores an n-dimensional reference featurevector. The classification may be done by mapping event data to neuronreference data so that each neuron describes part of a decision space.When training such a solution, a set of neurons may be defined given thetraining data so that the entire decision space is represented by allneurons. Given the training data relevant to a single user/individual,the set of neurons to represent the decision space can be reasonablysmall, which may in turn reduce the memory space, die size, and powerrequirements of the hardware solution. Thus, the cost impact, powerconsumption and complexity of such a hardware solution may be minimalbecause the illustrated personal classifier 16 only handles the eventscorresponding to a single user 10 (e.g., rather than a largepopulation). The generation of the self-training data 20 and thetraining of the personal classifier 16 based on the self-training data20 may be considered a “self-training stage” of the device 12.

During a “validation stage” of the device 12, the generic classifiers 14and the personal classifier 16 may be used online, via arrows 26, togenerate validation data 28 based on a second plurality of events 18 bassociated with the device 12. If the validation data 28 indicates thatthe personal classifier 16 satisfies a confidence condition relative tothe generic classifiers 14, then the device 12 may automaticallysubstitute the personal classifier 16 for the generic classifiers 14when generating, via arrows 32, ongoing classification data 30 based ona third plurality of events 18 c. As will be discussed in greater detailbelow, the confidence condition may include an agreement thresholdbetween the generic classifiers 14 and the personal classifier 16 beingreached and a validation instance threshold being reached. Moreover, thedevice 12 may also conduct periodic verifications that the confidencecondition is satisfied and, if any of the periodic verifications areunsuccessful, re-train the personal classifier 16.

Turning now to FIG. 2, a method 34 of operating a device such as, forexample, the device 12 (FIG. 1), is shown. The method 34 may beimplemented in one or more modules as a set of logic instructions storedin a machine- or computer-readable storage medium such as random accessmemory (RAM), read only memory (ROM), programmable ROM (PROM), firmware,flash memory, etc., in configurable logic such as, for example,programmable logic arrays (PLAs), field programmable gate arrays(FPGAs), complex programmable logic devices (CPLDs), infixed-functionality logic hardware using circuit technology such as, forexample, application specific integrated circuit (ASIC), complementarymetal oxide semiconductor (CMOS) or transistor-transistor logic (TTL)technology, or any combination thereof. For example, computer programcode to carry out operations shown in method 34 may be written in anycombination of one or more programming languages, including an objectoriented programming language such as Java, Smalltalk, C++ or the likeand conventional procedural programming languages, such as the “C”programming language or similar programming languages.

Illustrated processing block 36 provides for using one or more genericclassifiers to generate self-training data based on a first plurality ofevents associated with the device. Block 37 may train a personalclassifier based on the self-training data. Block 37 may be conductedonline either on the device or remotely (e.g., on a cloud computinginfrastructure, with the training results being returned to the devicein real-time). Thus, block 37 might involve programming a silicon-basedpattern recognition neural network (e.g., mapping the personalclassifier to a hardware accelerator/classification engine) with a setof neurons reflecting the self-training data, wherein the formulation ofthe set of neurons for personal classifier may take place either on thedevice or on a cloud computing infrastructure (e.g., the devicetransmits the self-training data to the cloud and receives the neuronprogramming parameters from the cloud). Accordingly, the personalclassifier may be trained online for a single individual, whereas eachof the one or more generic classifiers may be trained offline for aplurality of individuals.

Although the personal classifier is mapped to a hardware accelerator inthe illustrated example, the generic classifier might not use a hardwareaccelerator in order to be cost effective. For the personalizedclassifier, on the other hand, the number of neurons may be small, andtherefore reasonable for use with a hardware accelerator.

The one or more generic classifiers and a personal classifier may beused at block 38 to generate validation data based on a second pluralityof events associated with the device, wherein a determination may bemade at block 40 as to whether a confidence condition is satisfied. Asalready noted, the confidence condition may include, for example, anagreement threshold between the one or more generic classifiers and thepersonal classifier being reached, a validation instance threshold beingreached, and so forth. If the validation data indicates that thepersonal classifier satisfies the confidence condition relative to theone or more generic classifiers, illustrated block 42 substitutes thepersonal classifier for the one or more generic classifiers.Substituting the personal classifier for the one or more genericclassifiers may result in only the personal classifier being used toautomatically generate ongoing classification data based on a thirdplurality of events, wherein the ongoing classification data may in turnbe used to perform other real-time operations such as, for example,cursor movements, application launches, networking communications, andso forth.

FIG. 3 shows a more detailed method 44 of operating a device such as,for example, the device 12 (FIG. 1). The method 44 may also beimplemented in one or more modules as a set of logic instructions storedin a machine- or computer-readable storage medium such as RAM, ROM,PROM, firmware, flash memory, etc., in configurable logic such as, forexample, PLAs, FPGAs, CPLDs, in fixed-functionality logic hardware usingcircuit technology such as, for example, ASIC, CMOS or TTL technology,or any combination thereof. In starting block 46, a new device (e.g.,“out-of-box”) equipped with a generic classifier may be acquired by auser. The generic classifier may be used at block 48 to determine thecategory of each data instance (e.g., categorize each event), which maythen be added to a personal data pool maintained for each data category.

Thus, if the device is, for example, a wristwatch containing motionsensing technology (e.g., one or more accelerometers) configured todetect three different hand gestures, block 48 might involve creatingand maintaining a personal data pool for each category of hand gesture.Accordingly, each time a particular hand gesture is detected, theunderlying event data (e.g., accelerometer data) corresponding to thatevent may be added to the personal data pool corresponding to the handgesture in question. Illustrated block 50 determines whether the numberof data instances/events in each category of data is larger than aparticular threshold (“D_(thresh)”). If not, the method 44 may repeatblock 48. If so, the personalized classifier may be trained with thecollected personal data at block 52 and the self-training stage maycomplete.

Illustrated block 54 begins the validation stage by initializing one ormore variables such as, for example, the number of instances/events(e.g., N=0) and the number of instances/events for which the genericclassifier and the personal classifier agree (e.g., N_(agree)=0). For anew data instance, block 56 may run both the generic classifier and thepersonalized classifier, and increment the number of instances/events(e.g., N=N+1). Additionally, illustrated block 58 determines whether thegeneric classifier agrees with the personalized classifier (e.g., bothclassifiers categorize the event as the same gesture). If not, theillustrated method 44 may repeat block 56. If so, the number ofinstances/events for which the generic classifier and the personalclassifier agree may be incremented (e.g., N_(agree)=N_(agree)+1) atblock 60.

Additionally, a determination may be made at block 62 as to whether avalidation instance threshold (“N_(min)”) has been reached (e.g.,N>N_(min)). If not, the illustrated method 44 may repeat block 56. Ifso, a determination may be made at block 64 as to whether an agreementthreshold (“Amin” percentage) between the generic classifier and thepersonal classifier has been reached (e.g., N_(agree)/N>A_(min)). Ifnot, the self-training of the personal classifier may be repeated byreturning the process to block 48. Otherwise, the personal classifierhas passed the validation stage and illustrated block 66 sets arecalibration timeout to a particular value (e.g.,T_(recal)=CurrentTime+T). If it is determined at illustrated block 68that the recalibration timeout has expired (e.g., CurrentTime>Trecal), aperiodic verification that the confidence condition is satisfied may beconducted by returning the process to block 54 and the validation stage.If the recalibration timeout has not expired, termination block 70 mayuse the personalized classifier to categorize ongoing events online andin real-time.

Turning now to FIG. 4, a logic architecture 72 (72 a-72 e) is shown. Thearchitecture 72 may generally implement one or more aspects of a methodsuch as, for example, the method 34 (FIG. 2) and/or the method 44 (FIG.3), already discussed. Accordingly, the architecture 72 may be anapparatus readily incorporated into a device such as, for example, thedevice 12 (FIG. 1), already discussed. Thus, the architecture 72 maygenerally include one or more generic classifiers 72 a and a personalclassifier 72 b. In the illustrated example, the one or more genericclassifiers 72 a generates self-training data based on a first pluralityof events associated with the device, wherein a trainer 72 c coupled tothe personal classifier 72 b may train the personal classifier based onthe self-training data. The trainer, which may reside on the device, ona cloud computing infrastructure, and so forth, may also map thepersonal classifier to a hardware accelerator (e.g., programmablepattern recognition neural network) having an accelerated responsivenessthat is suitable for real-time training and classifications.

Additionally, the one or more generic classifiers 72 a and the personalclassifier 72 b may generate validation data based on a second pluralityof events associated with the device. In the illustrated example, avalidator 72 d coupled to the one or more generic classifiers 72 a andthe personal classifier 72 b substitutes the personal classifier 72 bfor the one or more generic classifiers 72 a if the validation dataindicates that the personal classifier 72 b satisfies a confidencecondition relative to the one or more generic classifiers 72 a. Asalready noted, the confidence condition may include an agreementthreshold between the one or more generic classifiers 72 a and thepersonal classifier 72 b being reached, a validation instance thresholdbeing reached, and so forth. Thus, each of the one or more genericclassifiers 72 a may be trained offline for a plurality of individualsand the personal classifier 72 b may be trained online for a singleindividual. In one example, the generic classifiers 72 a are implementedin software with a relatively high level of complexity and the personalclassifier 72 b is implemented in hardware with a relatively low levelof complexity due to the focus of the personal classifier 72 b on asingle individual.

The illustrated architecture 72 also includes a confidence verifier 72 eto conduct a periodic verification that the confidence condition issatisfied. If the periodic verification is unsuccessful, the trainer 72c may re-train the personal classifier 72 b.

FIG. 5 shows a mobile device 74. The mobile device 74 may be part of aplatform having computing functionality (e.g., PDA, notebook computer,tablet computer), communications functionality (e.g., wireless smartphone), imaging functionality, media playing functionality (e.g., smarttelevision/TV), wearable functionality (e.g., watch, eyewear, headwear,footwear, jewelry) any combination thereof (e.g., MID). In theillustrated example, the device 74 includes a battery 76 to supply powerto the device 74 and a processor 78 having an integrated memorycontroller (IMC) 80, which may communicate with system memory 82. Thesystem memory 82 may include, for example, dynamic random access memory(DRAM) configured as one or more memory modules such as, for example,dual inline memory modules (DIMMs), small outline DIMMs (SODIMMs), etc.

The illustrated device 74 also includes a input output (IO) module 84,sometimes referred to as a Southbridge of a chipset, that functions as ahost device and may communicate with, for example, one or more userinterface (UI) devices 86 (e.g., touch screens, displays, keypads,sensors) and mass storage 88 (e.g., hard disk drive/HDD, optical disk,flash memory, etc.). The illustrated processor 78 may execute the logicarchitecture 72 that is configured to use one or more genericclassifiers to generate self-training data based on a first plurality ofevents associated with the device 74, use the one or more genericclassifiers and a personal classifier to generate validation data basedon a second plurality of events associated with the device 74, andautomatically substitute the personal classifier for the one or moregeneric classifiers if the validation data indicates that the personalclassifier satisfies a confidence condition relative to the one moregeneric classifiers, as already discussed. The logic architecture 72 mayoptionally be implemented external to the processor 78. Additionally,the processor 78 and the IO module 84 may be implemented together on thesame semiconductor die as a system on chip (SoC).

Moreover, the illustrated device 74 may also include a hardwareaccelerator 87 (e.g., programmable pattern recognition neural network)having an accelerated responsiveness that is suitable for real-timetraining and classifications. For example, one such hardware solutionmight include a silicon implementation of an associative memory havingan array of neurons, wherein each neuron stores an n-dimensionalreference feature vector.

ADDITIONAL NOTES AND EXAMPLES

Example 1 may include a wearable device comprising a battery to powerthe device, a user interface to detect a first plurality of eventsduring a self-training stage of the wearable device and a secondplurality of events during a validation stage of the wearable device,one or more generic classifiers to generate self-training data based onthe first plurality of events, a personal classifier, wherein the one ormore generic classifiers and the personal classifier are to generatevalidation data based on the second plurality of events, and a validatorcoupled to the one or more generic classifiers and the personalclassifier, the validator to substitute the personal classifier for theone or more generic classifiers if the validation data indicates thatthe personal classifier satisfies a confidence condition relative to theone or more generic classifiers.

Example 2 may include the wearable device of Example 1, wherein each ofthe one or more generic classifiers is trained offline for a pluralityof individuals and the personal classifier is to be trained online for asingle individual.

Example 3 may include the wearable device of Example 1, wherein theconfidence condition is to include an agreement threshold between theone or more generic classifiers and the personal classifier beingreached.

Example 4 may include the wearable device of Example 3, wherein theconfidence condition is to further include a validation instancethreshold being reached.

Example 5 may include the wearable device of any one of Examples 1 to 4,further including a trainer coupled to the personal classifier, thetrainer to train the personal classifier based on the self-trainingdata.

Example 6 may include the wearable device of Example 5, furtherincluding a confidence verifier to conduct a periodic verification thatthe confidence condition is satisfied, wherein the trainer is tore-train the personal classifier if the periodic verification isunsuccessful.

Example 7 may include the wearable device of Example 5, furtherincluding a hardware accelerator, wherein the trainer is to map thepersonal classifier to the hardware accelerator.

Example 8 may include a method of operating a device, comprising usingone or more generic classifiers to generate self-training data based ona first plurality of events associated with the device, using the one ormore generic classifiers and a personal classifier to generatevalidation data based on a second plurality of events associated withthe device, and substituting the personal classifier satisfies aconfidence condition relative to the one or more generic classifiers.

Example 9 may include the method of Example 8, wherein each of the oneor more generic classifiers is trained offline for a plurality ofindividuals and the personal classifier is trained online for a singleindividual.

Example 10 may include the method of Example 8, wherein the confidencecondition includes an agreement threshold between the one or moregeneric classifiers and the personal classifier being reached.

Example 11 may include the method of Example 10, wherein the confidencecondition further includes a validation instance threshold beingreached.

Example 12 may include the method of any one of Examples 8 to 11,further including training the personal classifier based on theself-training data, and mapping the personal classifier to a hardwareaccelerator.

Example 13 may include the method of Example 12, further includingconducting a periodic verification that the confidence condition issatisfied, and re-training the personal classifier if the periodicverification is unsuccessful.

Example 14 may include at least one computer readable storage mediumcomprising a set of instructions which, when executed by a computingdevice, cause the computing device to use one or more genericclassifiers to generate self-training data based on a first plurality ofevents associated with the computing device, use the one or more genericclassifiers and a personal classifier to generate validation data basedon a second plurality of events associated with the device, andsubstitute the personal classifier for the one or more genericclassifiers if the validation data indicates that the personalclassifier satisfies a confidence condition relative to the one or moregeneric classifiers.

Example 15 may include the at least one computer readable storage mediumof Example 14, wherein each of the one or more generic classifiers is tobe trained offline for a plurality of individuals and the personalclassifier is to be trained online for a single individual.

Example 16 may include the at least one computer readable storage mediumof Example 14, wherein the confidence condition is to include anagreement threshold between the one or more generic classifiers and thepersonal classifier being reached.

Example 17 may include the at least one computer readable storage mediumof Example 16, wherein the confidence condition is to further include avalidation instance threshold being reached.

Example 18 may include the at least one computer readable storage mediumof any one of Examples 14 to 17, wherein the instructions, whenexecuted, cause the computing device to train the personal classifierbased on the self-training data, and map the personal classifier to ahardware accelerator.

Example 19 may include the at least one computer readable storage mediumof Example 17, wherein the instructions, when executed, cause thecomputing device to conduct a periodic verification that the confidencecondition is satisfied, and re-train the personal classifier if theperiodic verification is unsuccessful.

Example 20 may include an event categorization apparatus comprising oneor more generic classifiers to generate self-training data based on afirst plurality of events associated with a device, a personalclassifier, wherein the one or more generic classifiers and the personalclassifier are to generate validation data based on a second pluralityof events associated with the device, and a validator coupled to the oneor more generic classifiers and the personal classifier, the validatorto substitute the personal classifier for the one or more genericclassifiers if the validation data indicates that the personalclassifier satisfies a confidence condition relative to the one or moregeneric classifiers.

Example 21 may include the apparatus of Example 20, wherein each of theone or more generic classifiers is trained offline for a plurality ofindividuals and the personal classifier is to be trained online for asingle individual.

Example 22 may include the apparatus of Example 20, wherein theconfidence condition is to include an agreement threshold between theone or more generic classifiers and the personal classifier beingreached.

Example 23 may include the apparatus of Example 22, wherein theconfidence condition is to further include a validation instancethreshold being reached.

Example 24 may include the apparatus of any one of Examples 20 to 23,further including a trainer coupled to the personal classifier, thetrainer to train the personal classifier based on the self-training dataand map the personal classifier to a hardware accelerator.

Example 25 may include the apparatus of Example 24, further including aconfidence verifier to conduct a periodic verification that theconfidence condition is satisfied, wherein the trainer is to re-trainthe personal classifier if the periodic verification is unsuccessful.

Example 26 may include an event categorization apparatus comprisingmeans for performing the method of any of Examples 8 to 13.

Thus, techniques described herein may enable a user of a device to startwith complex software based generic classifiers, wherein a constrainedhardware based personal classifier may play a larger role incategorizing events over a longer time span as the personal classifierlearns the behavioral patterns of a particular individual. Such anapproach may reduce data variations, which may in turn enable minimalfeatures to be learned, less computation in feature extractions andeasier classification tasks. Moreover, use of a hardware based personalclassifier may result in fewer hardware components, smaller die size andimproved power efficiency. Simply put, techniques described herein mayenable longer battery life, higher accuracy due to better dataseparation between categories, and improved speed of recognition.

Embodiments are applicable for use with all types of semiconductorintegrated circuit (“IC”) chips. Examples of these IC chips include butare not limited to processors, controllers, chipset components,programmable logic arrays (PLAs), memory chips, network chips, systemson chip (SoCs), SSD/NAND controller ASICs, and the like. In addition, insome of the drawings, signal conductor lines are represented with lines.Some may be different, to indicate more constituent signal paths, have anumber label, to indicate a number of constituent signal paths, and/orhave arrows at one or more ends, to indicate primary information flowdirection. This, however, should not be construed in a limiting manner.Rather, such added detail may be used in connection with one or moreexemplary embodiments to facilitate easier understanding of a circuit.Any represented signal lines, whether or not having additionalinformation, may actually comprise one or more signals that may travelin multiple directions and may be implemented with any suitable type ofsignal scheme, e.g., digital or analog lines implemented withdifferential pairs, optical fiber lines, and/or single-ended lines.

Example sizes/models/values/ranges may have been given, althoughembodiments are not limited to the same. As manufacturing techniques(e.g., photolithography) mature over time, it is expected that devicesof smaller size could be manufactured. In addition, well knownpower/ground connections to IC chips and other components may or may notbe shown within the figures, for simplicity of illustration anddiscussion, and so as not to obscure certain aspects of the embodiments.Further, arrangements may be shown in block diagram form in order toavoid obscuring embodiments, and also in view of the fact that specificswith respect to implementation of such block diagram arrangements arehighly dependent upon the platform within which the embodiment is to beimplemented, i.e., such specifics should be well within purview of oneskilled in the art. Where specific details (e.g., circuits) are setforth in order to describe example embodiments, it should be apparent toone skilled in the art that embodiments can be practiced without, orwith variation of, these specific details. The description is thus to beregarded as illustrative instead of limiting.

The term “coupled” may be used herein to refer to any type ofrelationship, direct or indirect, between the components in question,and may apply to electrical, mechanical, fluid, optical,electromagnetic, electromechanical or other connections. In addition,the terms “first”, “second”, etc. may be used herein only to facilitatediscussion, and carry no particular temporal or chronologicalsignificance unless otherwise indicated.

Those skilled in the art will appreciate from the foregoing descriptionthat the broad techniques of the embodiments can be implemented in avariety of forms. Therefore, while the embodiments have been describedin connection with particular examples thereof, the true scope of theembodiments should not be so limited since other modifications willbecome apparent to the skilled practitioner upon a study of thedrawings, specification, and following claims.

We claim:
 1. A wearable device comprising: a battery to power thewearable device; a user interface to detect a first plurality of eventsduring a self-training stage of the wearable device and a secondplurality of events during a validation stage of the wearable device;one or more generic classifiers to generate self-training data based onthe first plurality of events; a personal classifier, wherein the one ormore generic classifiers and the personal classifier are to generatevalidation data based on the second plurality of events; and a validatorcoupled to the one or more generic classifiers and the personalclassifier, the validator to substitute the personal classifier for theone or more generic classifiers if the validation data indicates thatthe personal classifier satisfies a confidence condition relative to theone or more generic classifiers.